Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints

This paper presents a constrained adaptive dynamic programming (CADP) algorithm to solve general nonlinear nonaffine optimal control problems with known dynamics. Unlike previous ADP algorithms, it can directly deal state constraints. Firstly, generalized policy iteration (CGPI) framework is developed handle constraints by transforming the traditional improvement process into optimization problem. Next, we propose an actor-critic variant of CGPI, called CADP, in which both and value functions are approximated multi-layer neural networks map system states inputs function, respectively. CADP linearizes problem locally quadratically linear problem, then obtains update network solving its dual A trust region constraint added prevent excessive update, thus ensuring linearization accuracy. We determine feasibility calculating minimum boundary using two recovery rules when infeasible. The vehicle path-tracking task used demonstrate effectiveness this proposed method.